Is there A Tribe Called Quant?

A non-positivist take on quantitative methods

There are a lot of aspects in the application of quantitative methods that require a philosophical judgment from the researcher – indeed far more than what most positivist scholars tend to acknowledge or even think of (see Gorard, 2006). When conducting a form of quantitative text analysis, for instance, we might question the ontological status of a given corpus, that is, who are the social actors that produced it, under which circumstances, for what purposes, and what are its links to as well as its effects on reality. Depending on our answers to these questions, among others, our way of applying this method and evaluating its empirical results will vary quite significantly, from the strong objectivist stance of a positivist scholar to the equally strong relativist stance of a post-structuralist one. The same is true for the epistemic value we attribute to the researcher’s codebook: does it “inhere” from or “apply” to the textual data in a relatively straightforward and unproblematic manner, or is there some process of social construction that mediates the relation between the research codes and the text corpus, as the critical realist or social constructivist would argue? Similar epistemological questions could be raised when it comes to the researcher’s design of surveys and the evaluation of their results. As for statistical modeling, there are countless discussions to be had concerning the epistemic value of the data used in statistical modeling. The extent to which a researcher agrees that ‘sociological variables’ are ‘observed manifestations’ of ‘uncertain and generally unobservable (but nonetheless theorisable) data generating processes’ (Babones, 2015: 459-460), for instance, will render their application of statistical models either more positivist or realist/interpretivist in character.

So there is no single epistemological and ontological stance (i.e. positivism) that researchers have to adopt towards quantitative methods in principle, even if positivist assumptions happen to permeate most quantitative research communities in practice. But we could take things even further and make the reflexive claim that various quantitative researchers who might consider themselves positivists do not truly adopt a strict positivist stance towards their work. This effectively mirrors Carla Willig’s (2016) argument that most self-described “constructivist” and “qualitative” scholars tacitly adhere to ontological realism in their empirical research. The claim I am making here is that a number of researchers who draw on quantitative methods (un)consciously subscribe to certain non-positivist assumptions, and more specifically to realist assumptions (for a concise and accessible introduction to scientific/critical realism, see Patomäki and Wight, 2000; Maxwell, 2012: 1-69; Dowding, 2016: 9-36).

For instance, Paul MacDonald (2003) has shown that some rational choice theorists adopt scientific realist as opposed to instrumental empiricist assumptions towards the ontological nature of the empirical phenomenons under investigation. Likewise, the increasing acceptance among quantitative researchers that causality requires statistical correlation and the theoretical specification of causal mechanisms, which are partially unobservable but nevertheless real, points to the erosion of a core positivist tenet in favour of a more realist stance (Johnson, 2006; Bennett, 2013: 205-230; Hay, 2017). And on the epistemological side of things, some well-known quantitative scholars espouse more critical and reflexive views toward the creation of factual knowledge. John Gerring (2012) and Dimiter Toshkov (2016: 107-145), for instance, both eschew positivist beliefs in their accounts of what scientific description consists of. For them, empirical observations are the result of potentially difficult, problematic, and contentious inferences that are inextricably tied to our subjective (and quite often normative) viewpoint. Presumably, a number of quantitative scholars might agree with them given their influential position in this field of research. Yet their stance on the scientific description is a far cry from the positivist belief that empirical observations are objective and impersonal (since reality is deemed completely external to and unaffected by the social gaze of the researcher, there is a direct correspondence between the researcher and the reality under observation notwithstanding intentional “bias”). Such views liken them far more to scientific or critical realism, which combines ontological realism with epistemological interpretivism, than to the rehabilitated positivism of King, Keohane, and Verba (1994).

Alongside these principled and reflexive arguments, we can also make a more pragmatic one: there are scholars in the social sciences who make use of quantitative methods without considering themselves positivists or tying their research to positivist assumptions. Hence, it is conceivable and plausible to do so, regardless of how representative or salient their research happens to be. This is often the case for researchers who adopt the ‘mixed-methods’ paradigm (for a practical introduction to mixed-methods research, see Morgan, 2014). For instance, one paper on access to obstetric services in the Rohingya refugee camp of Bangladesh during the pandemic combined a large-N survey with in-depth interviews (Barua et al. 2022). Its clear emphasis on the subjective accounts of mothers (obtained through interviews) suggests that the beliefs and perceptions of actors are as important as the objective patterns tied to socio-material constraints (obtained through surveys) when analysing the social determinants of access to healthcare. So while the authors do not explicitly state the philosophical and theoretical assumptions that underpin their research, this usage of quantitative methods clearly points to either a realist or constructivist viewpoint as opposed to a positivist one (a similar example is found in Maxwell, 2012: 74-75). Besides mixed-methods research, there are also non-positivist research traditions that explicitly integrate quantitative methods into their empirical research and social theories. Two good examples are the usage of quantitative text analysis (or lexicometrics) in critical discourse studies drawing from post-structuralism, and of multiple correspondence analysis (MCA) among field theorists in the vein of critical realism.

Having said all of this, is there really “A Tribe Called Quant” as the title of the blog post states? My answer to this question is a resounding: no. Not all researchers who draw on quantitative methods have to adhere to positivist tenets in principle, as the first argument suggests, nor do all of them belong to a single “quantitative research” paradigm in practice, as the second and third arguments suggest. Bottom line: for those of us who are interested in quantitative methods without its positivist corollaries, we can go right ahead and make use of them in line with our own philosophical views. To put in the famous words of A Tribe Called Quest, one of the best hip-hop groups of the 1990s: “Can I kick it? Yes, you can!”

A sober view of the “qual-quant” divide

So what do we make of the divide between quantitative and qualitative research in the social sciences? The previous arguments posit that methods, whether quantitative or qualitative, are not inextricably wedded to a particular philosophical paradigm. This, however, goes against the ‘incommensurability’ thesis that a number of scholars, mostly on the qualitative side of this divide, have defended (Guba and Lincoln, 1985; Sale, Lohfeld, and Brazil, 2002; cf. Morgan, 2007). According to this popular view, quantitative and qualitative research are grounded in ‘radically different assumptions about the nature of reality and truth in paradigms like realism and constructivism’ which make it ‘impossible to translate or reinterpret research between these paradigms’ (Morgan, 2007: 58). So if our own claims go against this established viewpoint, that is, if quantitative and qualitative research do not truly constitute two discrete philosophical paradigms with clear-cut and opposing ontological and epistemological assumptions, then the question is: what is left of that distinction? In other words, what does qualitative and quantitative research still refer to? In my view, we should consider the divide between quantitative and qualitative research to be both analytical and pragmatic, without neglecting or minimising the pluralism of philosophical views that grounds our different usages of these methods.

 

The idea that we can distinguish quantitative and qualitative research analytically draws upon the famous article of Goertz and Mahoney (2006). The latter summarise their perspective as follows:

We prefer to think of the two traditions as alternative cultures. Each has its own values, beliefs, and norms. Each is sometimes privately suspicious or sceptical of the other though usually more publicly polite. … [We] tell a tale of these two cultures. We do so from the perspective of qualitative researchers who seek to communicate with quantitative researchers. Our goal is to contrast the assumptions and practices of the two traditions toward the end of enhancing cross-tradition communication. Like Brady and Collier (2004), we believe that qualitative and quantitative scholars share the overarching goal of producing valid descriptive and causal inferences. Yet, we also believe that these scholars pursue different specific research goals, which in turn produce different norms about research practices. Hence, we emphasize here to a greater degree than Brady and Collier the distinctiveness in basic goals and practices in the two traditions.

Goertz and Mahoney develop no less than ten analytical ‘criteria’ that differentiate between quantitative and qualitative research. A notable one is their distinct approaches to explanation (criterion n°1), insofar qualitative researchers tend to explain individual cases by looking exhaustively at all of the factors that produce a specific outcome (the ‘causes-of-effects’), while quantitative researchers tend to estimate the average effects of independent variables on a wider range of cases (the ‘effects-of-causes’).

This is a very compelling view of the divide since it echoes our intuition that different research methods can serve different purposes. Where I differ somewhat from Mahoney and Goertz (2006) is that we should emphasise, more explicitly than they do, the analytical pluralism not just between quantitative and qualitative research, but also within each cultural tradition. For instance, more positivist-minded qualitative researchers might still retain a ‘correlational’ view of causation (criterion n°3) on a smaller number of cases (criterion n°6); and more realist-minded quantitative researchers might not give all statistical observations equal importance a priori (criterion n°6) because of their theoretical assumptions (Babones, 2015), or might privilege statistical variation rather than a pattern of fit (criterion n°7) because of their purposeful interest in outliers (Maxwell, 2012: 49-51). As such, I very much agree with Mahoney and Goertz (2006) that the distinctive features of qualitative and quantitative research are analytical as well as methodological and certainly not philosophical in the sense of there being an incommensurable divide between the two research strands. However, we should not overlook the intra- as well as inter-pluralism between quantitative and qualitative research, which is linked to our different philosophical assumptions. In other words, methods are not free-floating entities that we can dissociate from our empiricist, positivist, realist, constructivist, post-structuralist, or pragmatic views, although they are not inextricably wedded to any particular philosophical paradigm.

The fact that there are irreducible philosophical differences (pluralism) in our usage of methods, however, does not mean we cannot make our approaches inter-communicable and mutually intelligible to varying extents (pragmatism). Granted, we draw on the same methods differently according to our own philosophical approaches (more realistically, the research traditions of the scientific communities we are embedded in). But these differences do not prevent us from establishing a greater or lesser degree of inter-comprehension which can reach across these practical and philosophical divides in the actual world of academic research, that is, in various concrete instances or social contexts of knowledge production (Morgan, 2007). So what differentiates us as researchers is not the method we use per se, but rather the areas of research and the communities of scholars we associate ourselves with. This will lead us to adopt distinct approaches toward our empirical inquiries in line with those researchers (in a quantitative-positivist lab, we are likely to pursue similar/compatible approaches, etc.), which will include specific choices and applications of quantitative and qualitative methods. The “qual-quant” divide, assuming it can still be considered a divide rather than a continuum given the emergence of mixed-methods approaches, is therefore less dichotomous and more amendable in practice according to pragmatist philosophers of the social sciences such as David Morgan and James Johnson (see above). In a nutshell, it is less debilitating and more promising than we are often made to think. Hopefully, these sorts of comforting conclusions will help us figure out a little better who we are and what we are doing as researchers in the social sciences.

By Édouard Hargrove PhD Candidate in Political Science Marie Skłodowska-Curie Actions (MSCA) Fellow Université libre de Bruxelles (ULB) and Université Panthéon-Sorbonne (Paris I)